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ABSTRACT 

A learning opportunity risk is defined as an absence 
o£ instruction or insufficient attention to pruliciency at an early 
grade of instruction in a subject matter which will generate serious 
learning problems in later grades. A method for identifying such 
risks has been derived from analysis of district-level Instructional 
Accomplishment Information (lAI) data bases. The method involves 
examination of patterns of instructional accomplishments in the first 
through sixth grades as they relate tc the scope, sequence, and 
emphasis of grade-by-grade instruction. Th&i operational elements of 
the method are: (1) learning opportunities — skill areas on which 
instructional time Is actually invested in a given subject matter at 
a given grade level; (2) instructional accomplishments — the mean 
percentage of student performance arrayed by skill are'is and school 
grades; and (3) learning opportunity risks — patterns of instructional 
accomplishments within and across grade levels that appear to depress 
subsequent instructional performance. The method is illustrated for 
the subjects of elementary school mathematics and reading in two 
rural southern school districts. (JD) 
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METHOD FOR ANALYZING DISTRICT LEVEL lAI DATA BASES TO IDENTIFY LEARNING 
OPPORTUNITY RISKS 

Patricia Mi lazzo, Aaron Buchanan, Adrlenne Escoe, and Richard Schutz 

Abstract 

A learning opportunity risk Is defined as an absence of Instruction 
or insufficient attention to a proficiency at an early grade of instruc- 
tion in a subject matter that generates serious learning problems in 
later grades. A method for identifying such risks is described, 
deriving from analysis of d i st r let- 1 eve 1 Instructional Accomplishment 
Information System data bases. The method is illustrated for the 
subjects of elementary school mathematics and reading. 
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MFTHOO FOR ANALYZING DISTRICT LEVEL lAI DATA BASES TO IDENTIFY LEARNING 
OPPORTUNITY RISKS 

Patricia Mi lazzo, Aaron Buchanan, Adrienne Escoe, and Richard Schutz 

Introduce ion 

The intention to diagnose learning difficulties Is pervasive to 
Instructional planning, delivery, and assessment, but the methods that 
are used are incomplete and o:=ten maladaptive. Conventional methods for 
Instructional diagnosis (e.g., interpretation of grade equivalent scores 
using norm referenced data, or skill mastery based on data from 
criterion-referenced tests) tend to illuminate shortcomings attributable 
to students, education personnel, and/or education agencies rather than 
to shed light on intended accomplishments and unintended risks generated 
as a consequence of the investment of instructional time to date. 
Instructional Accomplishment Information (lAl) systems provide instru- 
mentation for illuminating consequential relationships among learning 
intentions, opportunities, accomplishments, and risks. Information 
about instructional accomplishments contrasted with information about 
learning opportunities is the basis for a new and straightforward kind 
of analysis where discrepancies between instructional accomplishments 
and learning opportunities can often be interpreted as learning 
opportunity risks. 

The present report, dealing with the identification of learning 
opportunity risks, extends the methodological procedures for analyzing 
lAi district level data bases. The methodology is compatible with and 
complements the methodology described and illustrated in earlier reports 
(Mi lazzo, Buchanan, & Schutz, 198I; Behr 6 Bachelor, I98I; Mi lazzo, 
1981) dealing with the analysis of instructional accomplishments. 

The methodology to be presented is most conveniently described via 
a particular application growing out of the Cooperative School Improve- 
ment Program of Educational Laboratories and R&D Centers, a program 
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sponsored by the National Institute of Education and coordinated by the 
Council for Educational Development and Research. The data p'-esented 
are from the Implementation of a school improvement effort initiated by 
the Appalachia Education Laboratory and SWRL, collaborating with a State 
Department of Education and a major state university in the Deep South. 
Two rural districts with high concentrations of low SES students took 
the lead initiative in the improvement implementation. 

Although the emphasis of the present report is methodological, the 
substantive results are also interesting. It is commonly held that the 
school achievement of low SES students in the rural south is particu- 
larly weak, and there is a good amount of data on national achievement 
tests to support such a contention. However, a different and far less 
gloomy perspective Is obtained when the focus is placed on specific 
skills that students do or do not spend time learning and practicing. 
Data support a contention that SWRL has confirmed consistently In other 
situations: schools can demonstrate bona fide student accomplishments 
grade-by-grade In reading and mathematics on most of the critical 
building blocks for long-range skill development, as well as on otr;^3r 
important skills (see Hanson, Bailey, & Schutz, I98O; Ml lazzo 6 
Buchanan, 1981). Data consistently show that accomplishments are 
strongest where sets of materials, such as textbooks, concentrate 
instruction, and weakest where the concentration in these materials is 
thin and fragmented. No comprehensive set of materials is perfect. To 
the contrary, nearly all mathematics and reading textbook series used 
widely across the country have areas of concentration and '*thin spots" 
in comnon. (See Buchanan, 1979; Escoe, 1981 •) Textbooks generally pro- 
vide necessary and sufficient guidance for many or most students to 
learn some, but not all, skills well. 

There are serious problems in skill development in the elementary 
grades that have a long-range impact on achievement through the high 
school years. However, the actions necesorary to eliminate a number of 
these problems are remarkably straightforward. Mostly, they involve 
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small but highly consequential adjustments in instructional practices, 
and they are actions which schools can implement directly, over the 
short range, with a sure and swift effect on achievement. Most impor- 
tantly, the patterns and conclusions that are characteristic of the 
present sample of low income, rural, southern students generally reflect 
the broader sample of students from most parts of the United States who 
have participated in SWRL's Instructional Accomplishment Ir^'ormat Ion 
Inquiry over the years. So, what Is reported here as a specific Inquiry 
of a specific geographic region, has meaning for a much broader popula- 
tion of students and schools. Finally, this regional Iteration of a 
line of Inquiry affords the authors another opportunity to clarify and 
enhance the notion of school Improvement built upon solid Information 
bases. 

A description of the populations of students participating In the 
present Inquiry and descriptions of the assessment Instrumentation and 
administration procedures are provided in the Appendix, which readers 
interested in these technical aspects may wish to consult at this point. 

Operational Elements of the Method 

The method Involves examination of patterns of Instructional 
accomplishments as they relate to the scope, sequence, and emphasis of 
grade-by-grade instruction. 

Learning opportunities constitute skill areas on which 
instructional time is actually >nvested in a given subject matter at a 
given grade level. These skill ireas constitute the architectural 
framework of the lAI instruments Lhat are administered to students. 

Instructional accomplishments are expressed in terms of r^ean 
percentage of student performance arrayed by skill areas and school 
grades. 
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Learning opportunity risks are Identified by examining the pattern 
of instructional accomplishments within and across grade levels to 
determine structural deficiencies that appear to depress subsequent 
instructional performances. 

Not all deficiencies in instructional programs are critical. 
However, instructional flaws that occur at strategic points in the 
structure of K-6 instruct Ion-- in the supporting beams, if you will "" are 
critical. To push th'2 structural analogy c bit further, no one likes to 
live with weak construction in any part of a house. However, we can 
disregard weaknesses that appear during the course of the construction, 
if we can be confident that these apparent weaknesses will be eliminated 
during the later course of the construction and maintenance. It is not 
sensible to devote additional ''remedial** attention to such matters; they 
are not structural risks. On the other land, weakness in any supporting 
beam needs immediate attention; and the best opportunity to strengthen 
this part of the structure Is early in the building process, before too 
much additional construction overlays it. 

The Method Applied in Mathematics 

The charts on the next two pages represent a reduction of the 
comprehensive lAI data from the two rural southern districts partici- 
pating in this application. Skills categories typically found and 
emphasized in K-6 textbook series are listed in the left-most column, 
and the elementary grades are displayed across the top row. The cells 
in each chart describe 1) an overall performance level for each skill 
category and 2) particular information on the specific skills in each 
category that seem to explain the performance level. In a way, the 
charts are a '"blueprint" of mathematics instruction and achievement in 
the two districts, and they look very much alike. The fact is that most 
districts participating in lAI inquiry over the years have achievement 
patterns similar to these patterns, be they suburban, urban, or rural, 
pacific northwest, west, midwest, or southwest districts. 
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What are the serious structural weaknesses, I.e., the learning 
opportunity risks, In these two K-b blueprints? If we were to focus 
only on apparent deficiencies, i.e., "low" performances, then we would 
find them mostly in whe Intermediate grades 4, 5, and 6. So that may 
seem like tie most locical place to look for program weaknesses needing 
adjustment. It is not. Skills thet are taught and assessed in the 
intermediate grades have mar.y antecedents in the primary grades, rather 
than arising full blowf. In the intermediate grades. Many problems with 
low performance In grades k and 5 are cumulative ones that have devel- 
oped over several years. While there are surely actions that need to be 
taken at the point where a problem Is Identified In the intermediate 
grades, even the most Intense Instructional attention will most likely 
be inadequate here. Structural weakness Is least tolerable for skills 
that take 2 to 3 years to teach. Grade I may be too early to look for 
structural deficiencies In instruction that become apparent in low per- 
formances In the intermediate grades. On the other hand, weaknesses 
that are identified In grades 5 and 6 are prohably not ones that schools 
ought to attend to first, because they often relate strongly to learning 
opportunity risks that were made at earlier grade levels. Grades 2 
through 4, then, would seem to be the target years to look for and 
reduce such risks, particularly when the Interest is In long range and 
significant Improvement. This analysis will therefore focus on those 
grade levels In charts 1 and 2. 

The process of analyzing district level data Is straightforward. 
It produces a description of structural strengths and weaknesses common 
it- K- 6 textbooks series that are In widest use across the country. What 
the state of Instruction should or could be Is not the subject of this 
type of analysis. Given the scope, sequence, and emphasis of Instruc- 
tion as it is currently, the analysis identifies weaknesses in the 
architecture of the instruction which produce learning opportunity risks 
that can be removed or shored up to improve students' accomplishments. 
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Analysis begins by following each of the skills categories across 
several grade Jevels on each of the charts, but with particular atten- 
tion to grades 2, 3. and Several distinct characteristics become 
apparent: 

!• Some skills categories include skills that are more discrete in 
the way they build across grade levels than are others. (Com- 
putation is not discrete; what is taught at one grade level is 
rigorously dependent on something having been taught a year 
earlier.) For example, although measurement skills may have 
certain important prerequisites just like computation, there is 
greater likelihood that teachers will be able to adjust the 
instruction that students have not had, but that should have 
been in place previously. Instruction In geometry, given the 
way it is presented in most textbooks, can also be adjusted 
with a similar level of local effort* 

2. Somf) skills categories are clearly a reflection of others. For 
example, the development of problem solving skills, rightly or 
wrongly, is tied so closely to the development of computation 
skills step-by-step, that it often appears to have very little 
structure of Its own. Therefore, it Is never clear when look- 
ing at proficiency data on problem solving to what extent low 
performance is attributable to difficulty with the computation, 
not just at the point where performance is being assessed but 
at the point where learning was actually supposed to take 
place* 

3« Some skills categories build on a groundwork of grade-by-grade 
development, but the amount of lesson space dedicated to them 
is diminished and fragmented. Recognition of fractions, par- 
ticularly equivalent fractions, is a good example. Thin 
instruction on this kind of skill in grades 3 and k can seri- 
ously depress achievement in later grades. Often skills in 
this category have a real value independent of other skills, 
but just as important, they also serve as ''enabling'* skills to 
support the development of complex computation skills. How- 
ever, not all of these skills are of equal importance in terms 
of their effect on long range accomplishments. There are some 
skills that tend to atrophy as soon as instruction on them 
stops. For example, performance on computation with expanded 
place value forms for whole nun^ers Is strong in grades 2 and 3 
where computation is first being taught. The purpose is to 
make it easier for students to see what is happening to place 
values when they perform the regrouping traditionally known as 
**borrowing** and ''carry ing ,** but performance on this skill 
decline steadily as soon as instruction on it stops about the 
end of grade 3. 
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4. The computation skills categories iiwolving whole numbers and 
fractions are ones that most obviously build over 2 to 3 years 
in terms of instruction and achievement. Structural weaknesses 
in these matters are going to be the most apparent, and the 
action necessary to correct them In some ways requires the most 
uncomplicated commitment of time and resources. 

Relative to long^-range improvement in accomplishments, we have 
previously narrowed our search for learning opportunity risks to grades 

2 through 4. Now a second level of analysis will narrow our focus even 
more. We will also focus on the kinds of skills that fall under points 

3 and 4, not because skills under points 1 and 2 do not have a real 
importance of their own; of course they do. But skills in points 3 and 

4 have a long range structural significance in terms of K-6 mathematics 
instruction that provides the opportunity to make major improvements 
across several years without catastrophic changes in the structure of 
instruction. 



Grade 2 

Given this focus and the underlying logic that supports it, the 
analysis begins at grade 2. The first step is to look down the grade 2 
column for skill categories that show low performances, and then to fol-* 
low the pattern across grades 2, 3, and 4, determining how performance 
is either maintained or problems compounded at the higher grade levels. 
Three skills categories stand out in grade 2 on both charts with clearly 
depressed performances: 

• Multiplication facts with 0-5 as one factor . 

This category shows average per forman^^es of kb% on Chart 1 
and kO% on Chart 2 in the grade 2 column. There is no ques* 
tion that many students are not proficient in this skill by 
the middle of grade 2. Follow the skill into grades 3 and k 
and even By the middle of grade 3, performance has nearly 
doubled. An item-by-i tem analysis for this skill category 
showed that almost 80% of the third grade students could 
answer each item dealing with multiplication facts to 5- By 
mid-grade 4 more than 30% could do the simple multiplication 
facts. The data in these charts tell us that between the 
middle of grade 2 and the middle of grade 3$ most students in 
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both school districts in this analysis learn these facts. 
The conclusion Is that this skill probably receives suffi- 
cient Instructional treatment between the middle of grade 2 
and the middle of grade 3 to disqualify it as learning oppor- 
tunity risk. While there is room for local adjustments In 
instructional practices to improve grade 2 performance on 
multiplication facts to 5i given the nature of local p'iorl- 
ties, the skill does not seem to require any more critical 
attention in grade 2, particularly when one looks at 
performance during the next year. 

• Addition and Subtraction algorithms . 

Chart 1 shows an average performance of SS% and Chart 2 shows 
57% in grade 2. This time, however, the skill category does 
not show significant improvements in grade 3 and only slight 
improvements in grade A. It is not true that performance on 
these skills declines in grade ?, but the nature of addition 
and subtraction algorithms changes. Students werf; able to 
add and subtract 2^ and 3~dlgit numbers without regrouping in 
grade 2, They had great difficultly regrouping with subtrac- 
tion given 2-digit numbers in grade 2, and in grade 3 they 
are regrouping with 3-digit numbers. By grade ^, the addi- 
tion and subtraction algorithms are applied to 5- and 
6-dlgit numbers. Given this long range pattern, the low 
grade 2 performance does signal a serious weakness. There 
does not seem to be sufficient instruction between the middle 
of grade 2 and the middle of grade 3 to accommodate either 
the weak entry level on the addition and subtraction algo- 
rithms or the change from smaller to larger numbers In the 
higher grades. A close look at grade 2 of all of the widely 
adopted textbook series shows that even by the end of grade 
2, there is insuf f ic lent lesson space allocated to teaching 
the algorithms involving regrouping, particularly wirh sub- 
traction. There are no more than 2 to 5 lessons In any of 
the series, and the lessons occur In the last third to fourth 
of .iiost textbooks. So it is likely that many students never 
reach these lessons in grade 2 instruction. This constitutes 
a learning opportunity risk. 

• Recognition of fractions . 

In grade 2, the performances are 2k% on chart 1 and 25% on 
chart 2. Following the pattern through grades 3 and ^, and 
even further Into 5 and 6, it is clear that there is a seri- 
ous learning opportunity risk here. Most students in both 
districts (and in districts generally, for that matter) do 
not learn to recognize a unit fraction or to develop much 
proficiency with part/whole relationships. In grades 3 and k 
the nature of the skill changes again and students are deal- 
ing with common fractions, comparisons of fractions, and 
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equivalent fractions. Again, there seems to be a serious 
Instructional deficiency In grade 2, and Improvement at this 
early grade level should have a significant and positive 
impaction later achievement In grades 3 and ^. A look at 
Instruction on fractions in grade 2 shows that most of the 
Instruction on fractions In most programs occurs in the last 
third of the textbool^.s, and then there are typically about 2 
to 5 lessons dedicated to the skill. Many students may not 
get to this part of their textbook, and for most students 2 
to 5 lessons are probably not sufficient. In any case, by 
the middle of grade 3, almost no one can give a fraction for 
a region or set. 

The analysis at grade 2 has identified two learning opportunity 
risks; i.e., skills categories with long range Implications, where there 
Is genuine structural deficiency In Instruction that does not provide 
students an opportunity to build a knowledge base that will suppport 
Increasingly more complex content. Addition and subtraction with 
regrouping , and Recognition of fractions are two skill areas that are 
treated in fhe last half of most grade 2 textbooks, with 2 to 5 lessons 
apiece, and which change in nature from one grade to the next, showing a 
serious compounding of achievement problems in grade 3. 

Both of these risks could be reduced by 1) more lessons and 2) more 
attention to the specific errors that students make. Fortunately, grade 
2 Instruction does seem to have some "room" for 10 to 15 additional les- 
sons in mathematics. The instructional program at this level is not as 
packed with skills to be taught, as It is In grades ^-6. Furthermore, 
students' performances give us workable Insights Into the sorts of 
lessons that are needed. For example, students do not have sufficient 
practice in subtraction with renaming 2-dlglt numbers. More work on 
this algorithm using addition and subtraction facts to 15 as venicles 
(where most students are reasonably proficient in grade 2) should have 
an immediate impact on grade 2 performance and a longer range impact on 
grade 3 performance. Similar clues are evident in students' perfor- 
mances on fractions. Given a set or region with some parts shaded, 
students consistently give a fraction for the set by relating the shaded 
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parts to the unshaded parts instead of relating the shaded parts to the 
whole (including both shaded and unshaded parts). Lessons that focus on 
the relationships of parts to a whole should improve grade 2 perfor- 
mances and provide a firmer basis for grade 3* 



Grade 3 

The same type of analysis can be completed for grade 3. Looking 
doMn the grade 3 column, a number of skills are weak: 

• multiplication facts with 6-9 as one factor (Chart 1 = 44%, 
Chart 2 = 60%) 

• Division facts with 6-9 as divisor or quotient (Chart 1 = 36%, 
Chart 2 « 45%) 

• multiplication and division algorithms with 1-diglt 
multipliers/divisors (Chart 1 = 19%, Chart 2 = 29%) 

Most students in these two districts have not learned the 
hard multiplication and division facts by the middle of grade 
3; but by the middle of grade 4 most students can demonstrate 
high performances on this skill. If we were to use the same 
logic that we used with the easy multiplication and division 
facts in grade 2, then we would opt for some greater atten- 
tion to these hard facts in grade 3, but not qualify this 
skill as a structural deficiency in grade 3. But that 
recommendation is premature. More analysis is in order. A 
special stress exists in grades 3 and 4 that does not exist 
in grade 2. Students are expected to learn the hard multi- 
plication and division facts between the middle of grade 3 
and the middle of grade 4 . . . anc they generally do. At 
the same time they are expected to apply these facts to e 
algorithms involving 1- and 2-diglt multipliers, 1- and 
2-digit divisors, and regrouping . . . and they generally 
don't. Performances on the simple multiplication and divi- 
sion algorithms for these students, and for most third grade 
students, are at about the guess level. By the middle of 
grade 4, more students can compute with 1-digit multipliers 
and divisors, but most students are still learning to compute 
with 2-digit multipliers and divisors. 

How to relieve the stress? A good opportunity seems to 
reside in grade 3 with the hard multiplication and division 
facts. The stress in grade 4 can be relieved by finishing 
the hard multiplication and division facts by the end of 
grade 3, where there is room in the curriculum to add several 
lessons. Grade 3 instruction is still not as packed with 
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skills to teach as grade h is. If most students can do these 
facts by the end of grade 3, attention can focus In grade A 
on applying them. Because of the special structural stress 
created by not knowing the hard multiplication and division 
facts by the end of grade 3, we will Identify these skills as 
critically significant in grade 3> 

On the other hand, the simple multlpl Iclaton and division 
algorithms are clearly not learned by most third grade 
students In either of these districts (19% and 23% average 
performances). Is this a learning opportunity risk? 
Probably not directly. The algorithms require stable multi- 
plication and division facts, and most students have not yet 
developed stable facts. The tendency In most of the text- 
books used widely at grade 3 Is to introduce the simple 
algorithms at the end of grade 3, but primarily as a preview 
of the serious Instruction and learning that will occur early 
in grade k on the same skills involving 1-dlgit multipliers 
and divisors, and later on the more complex algorithms. 
Therefore, we will not qualify the matter as a learning 
opportunity risk. Schools may be able to partly improve stu- 
dents' achievement performances on the simple algorithms if 
they restrict the content to the easy facts, 1-5, which most 
students do know by the middle of grade 3* 

Addition and subtraction algorithms 

We noted In the grade 2 analysis that performances on the 
algorithms Involving larger numbers and regrouping, espe- 
cially with subtraction, were low for most third grade 
students in these two districts. If we believe our previous 
story on grade 2, there should be a serious improvement In 
grade 3 performance on these algorithms by way of improving 
grade 2 performance. Meanwhile, students currently in grade 
3 can receive regular instruction to improve their perfor- 
mances on the algorithms: More problems that use the 
better-learned subtraction facts (to 15), regrouping with 
2-dlglt numbers, etc. Therefore, we will not identify this 
skill as needing special attention at grade 3* 

Recognition and comparison of fractions 

The logic described above should be, and is, appropriate for 
this skill as well. However, the Instructional deficiency on 
recognition of fractions lo the most profound weakness In 
primary level instruction. Less than one fifth of the stu- 
dents in grade 3 could associate a common fraction with a set 
or region. All districts participating In lAI inquiry over 
the years demonstrate the same risk. Systematic practice at 
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thts grade level cannot do anything but Improve performance. 
Students particularly should benefit from systematic Instruc- 
tion regarding what a fraction Is not. It is not the ratio 
of a designated part of a whole (e.g., shaded dots) to an 
undesignated part of a whole (e.g., unshaded dots), but 
rather the ratio of the designated parts to ,an_ parts. With- 
out this basic understanding, students will generally not be 
able to understand or Identify equivalent fractions In grade 
A or the concepts that underlie addition and subtraction of 
fractions, which is introduced at this same grade level. 
This means that many students, most students really, will not 
learn the basic sl<ills needed to worl< and progress regularly 
through large areas of the curriculum in grades 4, 5 and 6. 
We'll identify the sl<ill as a learning opportunity risk mer- 
iting special attention at grade 3- For many districts, a 
partial solution involves simply assuring that all students 
worl< through the lessons that are provided in textbook pro- 
grams. There is some evIdenceTBuchanan & et al., 1976; 
Graeber, 1977) that lessons are often skipped in the primary 
grades. Again, there is usually time to complete the avail- 
able lessons on fractions provided in grade 3 textbooks, and 
even to add several additional lessons. There is no such 
t ime jy grade 4 . 



Measurement and problem solving also show unusually low performance 
levels, and they will need routine instructional attention at grade 3. 
However, remember that the logic which underlies the method excludes 
these types of skills as learning opportunity risks, given current 
priorities embodied in K-6 programs. 

Three skills categories, then, have been identified as learning 
opportunity risks at grade 3: 

- Multiplication facts with 6-9 as one factor, 

- division facts with 6-9 as divisor or quotient, and 
recognition of common fractions. 



Grade k 

According to our logic, grade k is the last place we will look here 
for instructional weaknesses that have long range implications for math- 
ematics achievement in the elementary grades. Looking down the grade k 
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column, many of the skills categories that show the lowest scores were 
previously identified in grades 2 and 3 as needing special attention* 
The argument was that over one to two years, students who have had the 
benefit of the recommended adjustments on the addition and subtraction 
algorithm in grade 2, multiplication and division facts in grade 3* and 
recognition of fractions in grades 2 and 3 will enter the fourth grade 
with more stable skills. That is, many of the skills categories that 
presently show low performances in fourth grade, w'M show the effect of 
the recommended improvement efforts after one and two years, so that 
they should not need more than routine instructional attention* There- 
fore, at this point in the structure of K-6 mathematics instruction, 
maintenance of routine instructional attention is recommended for add!- 
tion and subtraction algorithms , multiplication and division algorithms , 
and recognition and comparison of fractions . However, one '•new'' cate- 
gory enters the instructional program in grade ^, and students generally 
find it very difficult: 

• Addition and subtraction wi t h f ract ions 

Students in both districts were most successful with addition 
involving common denominators. Looking across the skills 
category to grades 5 and 6, we can see that proficiency does 
not improve, particularly when the skill category expands to 
include mixed numerals. Given long range implications, the 
category needs systematic and additional attention in grade 
A, But we have already indicated that there are serious 
stresses in the grade k instructional program created by 
severe mismatches between expectations and available time. 
It is likely that schools will have much less opportunity to 
add several lessons to an already full school year, than in 
grades 2 and 3- There are partial adjustments, however, that 
are possible. Most students find the process of addition 
with fractions intuitively difficult, and a close look at 
performances on the lAI inventories gives us some insight 
into the kind of confusion that current instruction might 
focus on. A common error made by students is to add/subtract 
both numerators and denominators, especially when dealing 
with fractions with unlike denominators. (For example, 
students will often conclude that 3/^ + 5/6 equals 8/10.) 
Again, some attention to what not to do in addition and 
subtraction with fractions would be a useful complement to 
instruction that sets up the procedures that are appropr late. 
Also remember that students who enter grade 4 having 
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benefited from adjusted instruction in grades 2 and 3 on 
recognition of fractions should have a better understanding 
of fractions, especially equivalent fractions, which should 
take much of the current mystery out of the addition and 
subtraction processes. 



This basically concludes the mathematics analysis. There Is more 
Information to be obtained from an expanded analysis, but we have accom- 
plished what we set out to do: to identify a small number of skills 
categories in the elementary grades where minor adjustments In instruc- 
tion can significantly reduce learning opportunity risks and thereby 
produce important improvements In student performance, both in the imme- 
diate graJe level where the adjustment is made, and In the long range 
across several grade levels. 



The charts for reading are organized In the same way as the 
previous charts for mathematics. Chart 3» or District 1, contains the 
complete list of skills and all six elementary grades, since this Dis- 
trict participated in the Inquiry at all of the grades. District 2 does 
not teach reading as a formal subject area beyond grade A. For this 
reason. Chart A shows data for only those grades. 

We will apply the same logic and procedures here that were applied 
in the previous section: a focus on grades 2, 3, and and on skills 
that have long range implications for success in reading Instruction. 
But a first look down the columns for grades 2, 3 and A shows an Inter- 
esting finding that Is consistent with other data bases generated by 
previous lAI inquiry In reading: There are no skills categories where 
performances are dramatically depressed. In fact, if we were to attend 
to average performance levels only, accomplishments are everywhere very 
good. It would be Inadvisable to conclude from this finding that there 
are no learning opportunity risks in reading instruction at these grade 
levels and at later ones. However, the fact that the proficiency 
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patterns In reading are different from those in mathematics is notable. 
Weaknesses In the architecture of reading instruction are noi ascertain- 
able by simply scanning the skills categories and the associated 
proficiency patterns. Large parts of convent loncil reading programs, 
unlike mathematics programs, are not cumulatively structured, particu- 
larly after grade 2. After grade 2, students are spending less time 
learning technical reading skills, and more time applying them to mate- 
rial with Increasingly specialized lexicons, more complex syntax, longer 
texts, and more sophisticated messages. (See Fi ege-Kol Imann, 1978; 
Escoe, 1981.) Moreover, after grade 3, reading programs do not have the 
steady Influx of new skills that are found in mathematics. Conse- 
quently, it is not sensible to look for learning opportunity risks in 
reading by focusing on discrete, prerequisite skills as such. Instead, 
structural deficiencies are likely to be found between the cracks of the 
skills that students cycle through over several years. 

Grade 2 

Scanning down the grade 2 column, the first five categories pertain 
to decoding and morphology skills that do not extend directly into 
grades A through 6, either In terms of the volume of content In each 
skills category or In terms of the Impact on other skills categories. 
Presumably programs teach these skills so that students can perform the 
reading operation. A quick look across all grade levels supports the 
conclusion that proficiencies in these technical decoding skills are 
sufnclently stable, since most students are able to answer a variety of 
questions about the texts they decode. 

Similarly, word meaning looks strong on both charts. The most 
consistent difficulty seems to be an artifact of the performance format 
for antonyms: Students have a strong inclination to select the synonym 
for a word and not the antonym. Students do demonstrate difficulty when 
less coinnon words for their age/grade level are the target of assess- 
ment, and Instructional attention to expanded vocabulary should 
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generally improve students' learning opportunity. But the general 
category does not reflect a serious instructional defect. 



One sl<ins category shows a consistent difficulty on both Charts: 

• Story meaning 

Two genera] types of skills are assessed here, recognition of 
detail, and recognition of-main idea. Most participating stu- 
dents in both districts were able to answer direct who and what 
detail questions. However, less than half of the students in 
both districts were able to answer correctly items asking for 
the main idea or an appropriate title. We can follow this 
skills category across the grade levels: Grade 3--strong 
detail, weak main idea, drop down two categories to comprehen- 
sion: main idea and detail in grade ^--strong detail, weak 
main idea; grade 5**strong detail, main idea improving. That 
is a little late. While students do demonstrate an understand- 
ing of the straightforward detail in the texts they read, they 
do not seem to be able to "sum up" or capture the gist of what 
they have read. Widely used textbook series frequently ask 
students to select the main idea for a passage in their work- 
book activities. However, recommended practices for teaching 
this skill are ambiguous at best and there is little direct 
opportunity to learn the skill. (It is interesting that many 
strongly intended skills in reading are treated much the same 
way that similarly highly valued skills in mathematics are 
treated, for example problem solv ing--students are generally 
Just asked to "do it," to use their technical decoding or com- 
putation skills to "make inferences" or "solve problems.") 
Instruction on this skill seriously deficient and there are 
long range learning risks if the weaknesses are left unat- 
tended. Methods have been developed and used to teach this 
type of summarizing skill. Escoe (IS8l) has identified a 
series of applied strategies for systematic instruction and 
practice on main idea. The adjustments in practice are not 
major, but they do require commitment of instructional time and 
enhancement of teaching and practice materials. Currently, 
teachers simply lack the instructional resources to do the best 
Job of teaching main Idea. 



We shall find this same pattern and logic holding in grades 3 and ^ 
for several other highly valued intended reading skills. I'ioreover, the 
demarcation between many of these sk 1 1 1 s--sequence, cause/effect, main 
ldea--ls obscure . The skills are not discrete, which should Imply that 
instructional practices do not need to be discrete. Rather, a direct 
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instructional approach that teaches the dynamics of this collection of 
static abstractions would seem to be the economical and natural route to 
take. 



Grade 3 

The discussion for grade 2 on decoding, morphology, and word 
meaning can almost be repeated at grade 3^ But the story meaning 
category needs to be considered anew: 

• Story meaning 

Weak performance on main idea skills was again evident. On 
technical skills, for example decoding elements or computation 
facts, systematic attention to the skill in one grade usually 
mutes the urgency for special attention to the skill in the next 
grade level. But this expectation does not hold when treating 
Intended skills that students cycle through over several years. 
No doubt, performance In handling main ideas in grade 3 will 
benefit from systematic attention to the skill in grade 2, but 
material at the higher grade level involves more complex cogni- 
tive skills and a broader experience base. Systematic 
instruction aimed at sharpening and stablizing the skill is 
still critically important. Since instruction on this skill at 
grade 3 is weak or absent, this is a learning opportunity risk. 
This grade level is particularly significant because it is a 
conventional benchmark year. Hereafter, students will be 
spending a lot aore time reading to learn, not vice versa. 

The story meaning skills category at grade 3 includes another 
process sk 1 1 1 --sequence of events--that is not unrelated to main 
idea. Students who cannot order events temporally may not fully 
understand what they read, which can impede their ability to 
"sum up" a text (Escoe, 1981). Many students were not profi- 
cient on the sequence tasks. Almost kQ% of the students in 
District 1 could not answer a simple sequence of events question 
about a short story (e.g., "What happened f i rst in the story?" 
or "What happened last in the story?"). Less than S0% of the 
students could answer both of these questions correctly. 
Partly, performances are an artifact of trying to capture this 
type of skill in a paper and pencil mode. That is, there is 
always the ambiguity of "first before what" and "last after 
what." On the other hand, students probably have not been 
taught strategies for tracking story event.s, given typical 
teaching material. Systematic attention to structured texts, 
where temporal sequence is carefully organized to follow a logi- 
cal order, consistent with the general schooling experiences of 
many third graders, would h?lp. (See Stein, 1978.) Escoe 
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(1981) again has Identified a variety of strategies to support 
sequence of events instruction and practice. 

Grade k 

There is little more to add to the story at the grade k level. The 
learning opportunity risks apparent in grades 2 and 3 are evident in 
grade Main idea , sequence of events a and a related sl<ill mal<i ng 
predictions show weak performance levels. But again, widely used 
instructional materials intended for the fourth grade provide little 
direct guidance to teachers In terms of instructional practices. 
Rather, students are asked to apply the skills by answering questions 
about main idea, sequence, and probable occurrences. 

In sum^ elimination of the learning opportunity risks identified 
foi reading should prove easy in practice, if comple)^ in principle. 
That's a welcome switch in the typical relationship between research and 
use. Exhorting teachers to spend ''time on task" teachnq the "compre- 
hension process" skills as independent proficiencies is unnecessary and 
misguided. The learning opportunity risk is not to be found in euier 
the skills categories or In the "sum of the parts." Moreover, materials 
to operat ional ize the "task" are currently not now available in widely 
used instructional materials, and learning opportunity risk appears 
attributable to that fact. 

It is easy to show for reading how conditions that create risks can 
arise. In the conventional way that reading text series are prepared, 
devoting attention to "process" skills can be expensive in terms of both 
production and use costs. The effort involved in generating instruc- 
tional passages is tedious and time-consuming compared to the effort 
involved in generating word- and sentence-length units. And Instruc- 
tional passages, even when short, require a fair amount of page space. 
Moreover, for each passage, it is usually feasible to generate several 
questions about details in the text, but it is seldom feasible to con- 
struct more than one or two sequence-of-event questions and a single 
title/main idea question for the passage. 
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The specific activity entailed in eliminating learning opportunity 
risks in reading does warrant explicit attention. We will discuss that 
matter in the next section. The happy finding here Is that in terms of 
the structure of reading instruction, the adjustments are equivalent to 
"fine tuning" that can be effected over the short run. The elimination 
of a risk can be accomplished without heavy retraining of teachers, 
heavy retooling of materials, or heavy readjusting of any other aspects 
of the prevailing instructional environment. 

Deriving Action from the Analysis 

Findings yielded by the application of the method lead to 
actionable conclusions, to activity based upon the conclusions to elim- 
inate or circumvent the learning opportunity risks that have been 
Identified. Such activity is the proof of the pudding, and It is of 
course In part situational ly specific. However, it Is possible to set 
forth some general "do's and don'ts." We shall first consider the 
"don'ts." 

The conventional action would be to convert the risks Into a list 
of "objectives" or "domains," and to expect teachers to teach them. 
Don't. It's a simple action, but too simple. 

One step removed would be to provide teachers with curriculum 
guides where there are one or two example lessons. Teachers are then 
expected to develop parallel lessons in order to teach a specified set 
of skills. Don't. Although teachers can and do develop materials for 
use in their classrooms, for many teachers their teaching and record 
keeping responsibilities leave little time for development efforts. 

A third common action would be to provide teachers with an Index 
showing where particular skills are represented in several different 
textbook series and/or other Instructional materials. Although the 
principle of indexing has potential utility in classroom practice, It 
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too often proves to be unwieldly for routine Instruction, and there are 
practical, pedagogical problems. For example, a skimpy Instructional 
treatment In several different places is not the Instructional resource 
that is in order. Again, don't. 

These three categories of "don't" actions should be sufficient to 
make the general point: Time is a precious and fixed commodity for 
classroom teachers. When an action plan relies on "free" teacher time 
to develop and/or pull together large portions of Instructional 
resources, the resulting burden in terms of stress on teachers' oppor- 
tunity to teach is expensive and the actual cost will certainly outweigh 
the intended benefits. 

The school district level, rather than teacher or individual school 
level, is the reasonable focus for initiatives to eliminate learning 
opportunity risks. These agencies generally provide one or more compre- 
hensive textbook series. Teachers use these series as the backbone of 
their instructional programs. Where series are structurally weak, then 
the opportunity to teach is Inadvertently weak. The serious problems 
arise where the text mat', ials "fudge"; where they do not deal in a 
coherent, integrated mstr ' -clonal way with real stresses and strains In 
the K-6 sequence. Districts can obtain or generate coherent, integrated 
sets of lessons. Including instructional and practice activities, which 
teachers can then use to temper or circumvent the learning risks. 

Some subject matters are easier to accommodate than others. In 
mathematics, there is a great deal of redundancy built into a K-6 text- 
book series, and this feature can be put to good use. For example, in 
most series, attention to the hard multiplication facts in grade 4 is 
well integrated and reasonably thorough; but this is at grade 4, not at 
the end of grade 3, where our analysis has shown the need for additional 
lessons. The additional lessons already exist in most widely used text- 
book series, and placement is the problem. For skills like recognition 
of fractions, or main idea, the instructional treatment is likely to be 
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skimpy. However, the district may well have a resource center where 
we II -focused sequences of lessons can be found and organized for timely 
use by Individual teachers. Moreover, supplemental teaching modules are 
available commercially, and many of them provide systematic instruc- 
tional treatment of the kinds of skills identified earlier, even though 
they may not be organized in exactly the right way to meet every 
teacher's needs. 

The problem of fitting additional lessons Into the Instructional 
program requires explicit attention. It is easier to fit say 10 to 15 
additional lessons Into the earlier grades than the later grades; but It 
Is feasible to fit lessons into the upper grades, given district level 
attention to priorities for teaching and learning. It will be difficult 
wherever you try to fit additional Instruction Into a school year, but 
that does not make it any less critical to provide additional practice 
on skills that are key to students' future learning opportunity. 

Once Identified, there is nothing at all mysterious about a set of 
learning opportunity risks, and neither is there anything at all myster- 
ious about their elimination. It involves the provision of adequate 
lnstru..tional resources for teachers to use in their teaching to give 
students an opportunity to learn. Of such is the profession of 
educat Ion. 
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APPENDIX 



Population 

The mathematics Inquiry Involved a total of 1726 students, and the 
reading Inquiry 1185 students. Students attended two districts In one 
sou;'-«rn state, and they were In classrooms In five different schools. 
Approximately 1000 of the students participating In the mathematics 
Inquiry came from District 1, and 700 from District 2. Most of the 
students participating in the reading inquiry came from District 1, 
approximately 1000; and there were approximately 200 students from 
District 2. Intact classrooms were generally assessed. 

Administration Procedures 

The lAI inventories were administered in late January to all 
participating students. Teachers were asked to complete their adminis- 
tration of the inventories over a two week period. Because of the 
nature of mathematics instruction, students were not expected to have 
been taught al 1 of the content covered in the mid-year Inventories. 
Therefore, teachers were advised to assign only those sections of the 
inventory that generally represented instruction In mathematics 
completed by mid-year. Students were Instructed to skip entire, homo- 
geneous sections of the inathematics inventory. Reading inventories were 
different. Teachers were encouraged to administer the entire inventory 
at every grade level to all of thei> students. 

The inventories were not timed, and students were given a 



reasonable amount of time to answer all assigned sections of the Inven- 
tories. At grades 1, 2, and 3 , ^0 to 60 minutes were reasonable for 
each Inventory, At grades A-6, 60 to 70 minutes were acceptable. In 
grades 1 and 2, for both reading and mathematics, teachers were allowed 
to administer the assessment Item-by- Item to the entire class or to 
small groups, and students marked their answers directly on the booklet. 
Students were encouraged to ask questions whenever they were unsure 
about an Item format, picture cue, or direction. Teachers were told to 




27 



help students to understand what was being asked, as best they could 
without giving away the answer. In grades 3 through 6, inventories were 
self-admlnistered by students, who were allowed to asic teachers for help 
when an item format or direction was not understood. 

Scoring and Reporting 

Completed student materials were collected by teachers, checked for 
stray marks or other conditions that might interfere with electronic 
processing, batched and sent to a scoring center in Minneapolis. Book- 
lets and answer sheets were scanned by a Sentry model 70 optical mark 
reader. Scanner tapes were preprocessed, input records diagnosed, and 
data bases corrected where there may have been processing complications. 
Reports were generated by an IBM 3033 computer and returned to partic- 
ipating students, classrooms, schools, and districts. 

Instrumentation 

Definitive descriptions of the PVS instrumentation may be found in: 
Proficiency verification systems, 1981 Brief , SWRL Educational Research 
and Development, Los Alamitos, California; Designing an instruction- 
referenced system for large-scale evaluation of school achievement , 
Aaron D. Buchanan and Patricia A. Mi lazzo, presented at the Third Annual 
Conference of the California Society of Educational Program Auditors and 
Evaluators, Los Angeles, May 12-13, 1977; PVS operations manual , SWRL 
Educational Research and Development, Los Alamitos, California, 1978; 
Proficiency verification systems (PVS); Index of mathematical skills , 
Aaron D. Buchanan, Technical Note No. 3-76-01, SWRL Educational Research 
and Development, Los Alamitos, California, 1976; Proficiency verifica- 
tion systems (PVS); Mathematics test development , Jannine Perkins, 
Technical Note No. 3-76-03, SWRL Educational Research and Development, 
Los Alamitos, California, 1976; Consistency of items across mathematics 
Inventories of PVS , Jannine Perkins and Nancy Bennett, SWRL Educational 
Research and Development, Los Alamitos, California, 1978; Proficiency 
verification systems (PVS)t Index of reading skills , Lai la Fiege- 
Kollmann, Technical Note No. 3-77-01, SWRL Educational Research and 
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Development, Los Alamltos, California, 1977} Proficiency verification 
systems in reading , Lai la Fiege-Kol Imann, Technical Note No. 3-76-10, 
SWRL Educational Research and Development, Los Alamitos, California, 
1976; Proficiency verification systems; The development of an infor- 
mation system for second grade reading , Lai la Fiege-Kol Imann and 
Patricia Miiazzo, Technical Note No. 3-77-03, SWRL Educational Research 
and Development, Los Alamitos, California, 1977? Tryout of the reading 
prof 1 ciency ski 1 1 s inventory , Lai la Fi ege-Kol Imann, Technical Note No. 
3-78-11, SWRL Educational Research and Development, Los Alamitos, 
California, 1978. 
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